Dynamic Camera Field of View Adjustment

ABSTRACT

Various embodiments disclosed herein include techniques for adjusting the lateral position of a default field of view of a camera. The camera may change the lateral position of its field of view by creating relative movement between an image sensor and a lens of the camera, and may maintain the camera&#39;s field of view to the default field of view when the camera is stationary. In some embodiments, the lateral position may be set using orientation information associated with the camera. In other embodiments, the lateral position may be set using position information associated with a target object.

FIELD

This disclosure relates generally to dynamically adjusting a field ofview of a camera. More particularly, this disclosure relates toadjusting a field of view of a camera based on an orientation of thecamera.

BACKGROUND

Cameras continue to be an important feature of consumer electronicsdevices such as smartphones, tablets, and computers. These cameras areused for a wide range of operations, such as capturing videos or stillimages or facilitating live video conferencing sessions. During livevideo conferencing sessions, a user may want to share content fromdifferent portions of their environment, but this typically requires auser to precisely position and/or manually reposition the camera tocapture the desired content. Especially in instances where a user doesnot wish to actively hold the electronic device incorporating thecamera, this need to precisely position (or reposition) the electronicdevice may be burdensome to the user.

SUMMARY

The present disclosure relates to cameras, devices, and systems forsetting a default position of an adjustable field of view of a camera.In some embodiments, a system includes a device comprising a camera, thecamera having an optical axis and an adjustable field of view. Thecamera may include a lens, an image sensor, and a position control logicconfigured to: (i) obtain camera orientation information associated withthe camera, (ii) select a lateral position of a default field of viewusing the camera orientation information, (iii) control a relativeposition of the lens and the image sensor to maintain the adjustablefield of view as the default field of view while the camera isstationary, and (iv) capture an image at the default field of view.

In some instances, the system further includes an accessory devicecoupled to the device, and the camera orientation information includesrelative orientation information that includes a relative orientationbetween the camera and the accessory device. Additionally oralternatively, the camera orientation information includes relativeorientation information that includes a relative orientation between acamera and a surface identified in a scene surrounding the camera.Additionally or alternatively, the camera orientation informationincludes absolute orientation information that includes a relativeorientation between the camera and gravity.

The position control logic may be configured to perform optical imagestabilization while the camera is moving, during which the positioncontrol logic temporarily moves the adjustable field of view away fromthe default field of view in response to camera motion. In someinstances, the position control logic is (i) configured to obtainposition information associated with a target object, and (ii)configured to select the lateral position of the default field of viewusing the camera orientation information and the position information.In some of these instances, the position control logic is configured toidentify a set of potential lateral positions using the cameraorientation information. The position control logic selects, using theposition information, one of the potential lateral positions as thelateral position of the default field of view. In some instances, theposition control logic is configured to control a relative rotation ofthe image sensor around the optical axis by an amount determined usingthe camera orientation information.

Other embodiments include a camera having an optical axis and anadjustable field of view, in which the camera includes a lens, an imagesensor, and a position control logic. The position control logic isconfigured to: (i) obtain camera position information associated with atarget object; (ii) select a lateral position of a default field of viewusing the position information, (iii) control a relative position of thelens and the image sensor to maintain the adjustable field of view asthe default field of view while the camera is stationary, and (iv)capture an image at the default field of view. In some instances, theposition control logic is configured to a relative tilt between the lensand the image sensor using the position information.

In other instances, selecting the lateral position of the default fieldof view includes determining whether a set of candidate positions existsthat would position the target object in a first region of theadjustable field of view. In some instances, the position control logicis configured to select, in response to determining that the set ofcandidate positions exists, one of the set of candidate positions as thelateral position of the default field of view. In other instances theposition control logic is configured to select, in response todetermining that the set of candidate positions does not exist, thelateral position of the default field of view at a position that placesthe object in a second region of the adjustable field of view.

Still other embodiments include methods of capturing images. In someinstances, a method includes, at a system that includes a display and acamera having an adjustable field of view, capturing a first imagestream while the adjustable field of view has a default field of viewset at a first lateral position and generating a first set of outputimages from the first image stream. In some these instances, the firstlateral position may be selected using camera orientation informationassociated with the camera. In some instances, the first set of outputimages is a video feed, and the method further includes displaying, viathe display, a communication user interface for a videoconferencingsession, the communication user interface including the video feed. Thevideo feed may include a representation of a surface in a scenesurrounding the camera.

In some variations, the method includes capturing a second image streamwhile the adjustable field of view has the default field of view set ata second lateral position and generating a second set of output imagesfrom the second image stream, where the second lateral position isselected using position information associated with a target object.Additionally or alternatively, the first lateral position is selectedusing both the camera orientation information and position informationassociated with a target object. In some of these variations, the methodfurther comprises generating a second set of output images from thefirst image stream. The first set of output images may be generated froma first cropping boundary applied to the first image stream, and thesecond set of output images may be generated from a second croppingboundary applied to the first image stream.

In addition to the exemplary aspects and embodiments described above,further aspects and embodiments will become apparent by reference to thedrawings and by study of the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure will be readily understood by the following detaileddescription in conjunction with the accompanying drawings, wherein likereference numerals designate like structural elements, and in which:

FIGS. 1A and 1B show front and rear views, respectively, of an exampleof an electronic device having a camera with an adjustable field ofview. FIG. 1C depicts exemplary components of the device of FIGS. 1A and1B.

FIGS. 2A and 2D show side view of a variation of a camera as describedherein.

FIGS. 2B, 2C, 2E, and 2F show example images captured by the camera ofFIGS. 2A and 2D.

FIG. 3 shows a cross-sectional side view of an illustrative example of acamera that may be used with the devices described herein.

FIGS. 4A, 4D, 4F, and 4H show top views of a camera positioned to imagea person using an adjustable field of view. FIGS. 4B, 4E, 4G, and 4Ishow example images captured by the camera of FIGS. 4A, 4D, 4F, and 4H.FIG. 4C shows a front view of a device incorporating the camera of FIGS.4A, 4D, 4F, and 4H.

FIG. 5 depicts a method of setting a lateral position of a camera'sfield of view using position information associated with a targetobject.

FIGS. 6A and 6D show side views of a variation of a system as describedherein, including a device that has a camera with an adjustable field ofview. FIG. 6B shows an example image captured by the camera of FIG. 6A.FIG. 6C shows a front view of the device of FIG. 6A.

FIG. 7 depicts a method of setting a lateral position of a camera'sfield of view using orientation information associated with the camera.

FIG. 8A shows a side view of a variation of a system as describedherein, including a device that has a camera with an adjustable field ofview. FIG. 8B shows an example image captured by the camera of FIG. 8A.FIG. 8C shows a front view of the device of FIG. 8A.

FIGS. 9A and 9B show example images captured by a variation of thesystems described herein.

It should be understood that the proportions and dimensions (eitherrelative or absolute) of the various features and elements (andcollections and groupings thereof) and the boundaries, separations, andpositional relationships presented therebetween, are provided in theaccompanying figures merely to facilitate an understanding of thevarious embodiments described herein and, accordingly, may notnecessarily be presented or illustrated to scale, and are not intendedto indicate any preference or requirement for an illustrated embodimentto the exclusion of embodiments described with reference thereto.

Directional terminology, such as “top”, “bottom”, “upper”, “lower”,“front”, “back”, “over”, “under”, “above”, “below”, “left”, “right”,“vertical”, “horizontal”, etc. is used with reference to the orientationof some of the components in some of the figures described below, and isnot intended to be limiting. Because components in various embodimentscan be positioned in a number of different orientations, directionalterminology is used for purposes of illustration only and is in no waylimiting. The directional terminology is intended to be construedbroadly, and therefore should not be interpreted to preclude componentsbeing oriented in different ways. Also, as used herein, the phrase “atleast one of” preceding a series of items, with the term “and” or “or”to separate any of the items, modifies the list as a whole, rather thaneach member of the list. The phrase “at least one of” does not requireselection of at least one of each item listed; rather, the phrase allowsa meaning that includes at a minimum one of any of the items, and/or ata minimum one of any combination of the items, and/or at a minimum oneof each of the items. By way of example, the phrases “at least one of A,B, and C” or “at least one of A, B, or C” each refer to only A, only B,or only C; any combination of A, B, and C; and/or one or more of each ofA, B, and C. Similarly, it may be appreciated that an order of elementspresented for a conjunctive or disjunctive list provided herein shouldnot be construed as limiting the disclosure to only that order provided.

DETAILED DESCRIPTION

Reference will now be made in detail to representative embodimentsillustrated in the accompanying drawings. It should be understood thatthe following descriptions are not intended to limit the embodiments toone preferred embodiment. To the contrary, it is intended to coveralternatives, modifications, and equivalents as can be included withinthe spirit and scope of the described embodiments as defined by theappended claims.

The following disclosure relates to cameras having adjustable fields ofview, devices and systems incorporating the same, and methods of usingthe same to capture images. Specifically, the cameras, devices, andsystems may be used to set the default configuration of a camera's fieldof view (i.e., the configuration of the camera's field of view when thedevice is not moving). In some instances, a lateral position of thecamera's default field of view is set using position informationassociated with a target object in a scene surrounding the camera.Additionally or alternatively, the lateral position of the camera'sdefault field of view is set using orientation information associatedwith the camera. These and other embodiments are discussed below withreference to FIGS. 1A-9B. However, those skilled in the art will readilyappreciate that the detailed description given herein with respect tothese Figures is for explanatory purposes only and should not beconstrued as limiting.

The devices, systems, and methods described here include an electronicdevice having at least one camera with a moveable field of view. FIGS.1A-1C depict an example device 100 as described herein. FIG. 1A shows afront view of the device 100, which includes a display 102 and afront-facing camera 104. The display 102 may provide a graphical outputthat is viewable through or at a front exterior surface of the device100. The front-facing camera 104 is positioned to view a portion of theenvironment in front of the display 102. When the device 100 is used tofacilitate a live video conferencing session with a second device (notshown), the front-facing camera 104 may capture a video stream of theuser for transmission to a second device. A video stream received fromthe second device may be displayed via the display 102 to facilitatereal-time video communication between the two devices.

In some instances, the device 100 may further include a front-facingdepth sensor 106 that may calculate depth information for a portion ofthe environment in front of the device 100. The depth sensor 106 may beany suitable system that is capable of calculating the distance betweenthe depth sensor 106 and various points in the environment around thedevice 100. The depth sensor may generate a depth map including thesecalculated distances, some or all of which may be used in the varioustechniques described below. The depth information may be calculated inany suitable manner. In one non-limiting example, a depth sensor mayutilize stereo imaging, in which two images are taken from differentpositions, and the distance (disparity) between corresponding pixels inthe two images may be used to calculate depth information. In anotherexample, a depth sensor may utilize structured light imaging, wherebythe depth sensor may image a scene while projecting a known pattern(typically using infrared illumination) toward the scene, and then maylook at how the pattern is distorted by the scene to calculate depthinformation. In still another example, a depth sensor may utilize timeof flight sensing, which calculates depth based on the amount of time ittakes for light (typically infrared) emitted from the depth sensor toreturn from the scene. A time-of-flight depth sensor may utilize directtime of flight or indirect time of flight, and may illuminate an entirefield of coverage (i.e., the widest lateral extent to which the depthsensor is capable of providing depth information) at one time, or mayonly illuminate a subset of the field of coverage at a given time (e.g.,via one or more spots, stripes, or other patterns that may either befixed or may be scanned across the field of coverage).

FIG. 1B shows a rear view of the device 100, which includes a set ofrear facing cameras. In the variation shown in FIG. 1B, the set of rearfacing cameras includes a first rear-facing camera 108, a secondrear-facing camera 110, and a third rear-facing camera 112. Therear-facing cameras may have fields of view that at least partiallyoverlap with each other, which may allow the rear-facing cameras tocapture different aspects of a scene facing a rear surface of the device100. For example, in some instances each rear-facing camera has adifferent focal length, and thereby has a field of view with a differentsize. The choice of the size of a camera's field of view may impact thesituations in which a particular camera may be useful. For example,cameras with longer focal lengths (and narrower fields of view) areoften used in telephoto imaging where it is desirable to increase themagnification of a subject at farther distances, while cameras withshorter focal lengths (and wider fields of view) are often used ininstances where it is desirable to capture more of a scene (e.g.,landscape photography). It should be appreciated that the field of viewof a camera refers to the spatial extent of a scene that the camera isable to capture using an image sensor of the camera. As will bediscussed in more detail below, the camera and any associated devicesmay use all or only some of the camera's field of view when generatingimages. In other words, when the camera is used to capture an image, theimage presented to a user may only represent a subset of the camera'sfield of view.

Also shown there is a rear-facing depth sensor 114, which may beconfigured in any manner as discussed previously with respect to thefront-facing depth sensor 106. While the device 100 is shown in FIGS. 1Aand 1B as having four cameras and two depth sensors, it should beappreciated that the device 100 may have any number of cameras and depthsensors as desired. The principles described herein may be applied toany camera or cameras of the device 100. For the purpose ofillustration, the principles of operation described herein are describedwith respect to a single camera of a device, which may represent anycamera of that device (e.g., a front-facing camera, a rear-facingcamera, or the like).

In some embodiments, the device 100 is a portable multifunctionelectronic device, such as a mobile telephone, that also contains otherfunctions, such as PDA and/or music player functions. Exemplaryembodiments of portable multifunction devices include, withoutlimitation, the iPhone®, iPod Touch®, and iPad® devices from Apple Inc.of Cupertino, California. Other portable electronic devices, such aslaptops or tablet computers with touch-sensitive surfaces (e.g., touchscreen displays and/or touchpads), are, optionally, used. It should alsobe understood that, in some embodiments, the device is not a portablecommunications device, but is a desktop computer, which may have atouch-sensitive surface (e.g., a touch screen display and/or atouchpad). In some embodiments, the electronic device is a computersystem that is in communication (e.g., via wireless communication, viawired communication) with a display generation component. The displaygeneration component is configured to provide visual output, such asdisplay via a CRT display, display via an LED display, or display viaimage projection. In some embodiments, the display generation componentis integrated with the computer system (e.g., display 102). In someembodiments, the display generation component is separate from thecomputer system. As used herein, “displaying” content includes causingto display the content by transmitting, via a wired or wirelessconnection, data (e.g., image data or video data) to an integrated orexternal display generation component to visually produce the content.

FIG. 1C depicts exemplary components of device 100. In some embodiments,device 100 has a bus 126 that operatively couples I/O section 134 withone or more computer processors 136 and memory 138. I/O section 134 canbe connected to display 102, which can have touch-sensitive component130 and, optionally, intensity sensor 132 (e.g., contact intensitysensor). In addition, I/O section 134 can be connected withcommunication unit 140 for receiving application and operating systemdata, using Wi-Fi, Bluetooth, near field communication (NFC), cellular,and/or other wireless communication techniques. Device 100 can includeinput mechanisms 142 and/or 144. Input mechanism 142 is, optionally, arotatable input device or a depressible and rotatable input device, forexample. Input mechanism 142 is, optionally, a button, in some examples.Device 100 optionally includes various sensors, such as GPS sensor 146,accelerometer 148, directional sensor 150 (e.g., compass), gyroscope152, motion sensor 154, and/or a combination thereof, all of which canbe operatively connected to I/O section 134. Some of these sensors, suchas accelerometer 148 and gyroscope 152 may assist in determining anorientation of the device 100 or a portion thereof.

Memory 138 of device 100 can include one or more non-transitorycomputer-readable storage mediums, for storing computer-executableinstructions, which, when executed by one or more computer processors136, for example, can cause the computer processors to perform thetechniques that are described here (methods performed by the positioncontrol logics described below). A computer-readable storage medium canbe any medium that can tangibly contain or store computer-executableinstructions for use by or in connection with the instruction executionsystem, apparatus, or device. In some examples, the storage medium is atransitory computer-readable storage medium. In some examples, thestorage medium is a non-transitory computer-readable storage medium. Thenon-transitory computer-readable storage medium can include, but is notlimited to, magnetic, optical, and/or semiconductor storages. Examplesof such storage include magnetic disks, optical discs based on CD, DVD,or Blu-ray technologies, as well as persistent solid-state memory suchas flash, solid-state drives, and the like.

The processor 136 can include, for example, dedicated hardware asdefined herein, a computing device as defined herein, a processor, amicroprocessor, a programmable logic array (PLA), a programmable arraylogic (PAL), a generic array logic (GAL), a complex programmable logicdevice (CPLD), an application-specific integrated circuit (ASIC), afield-programmable gate array (FPGA), or any other programmable logicdevice (PLD) configurable to execute an operating system andapplications of device 100, as well as to facilitate setting a field ofview of a camera and capturing of images as described herein. Device 100is not limited to the components and configuration of FIG. 1C, but caninclude other or additional components in multiple configurations.

At least one camera 100 of device 100 is configured to controllablytranslate a position of its field of view by creating lateral relativemovement between an image sensor and a lens of the camera. FIG. 2A showsa schematic representation of a portion of a camera 200 that includes alens 202 and an image sensor 204. The lens 202, while represented inFIG. 2A as a single lens element, includes one or more lens elementsthat are collectively configured to direct light received by the camera200 toward the image sensor 204. The image sensor 204 receives lightfrom the lens 202 to capture images during one or more photography modes(e.g., a photo mode that can capture still images, a video mode that maycapture videos, a panoramic mode that can capture panoramic photos, aportrait mode that can capture a still photo having an artificial bokehapplied, or the like).

The camera 200 includes one or more actuators (not shown) that createrelative movement between the image sensor 204 and the lens 202, andthis relative movement may be controlled by a position control logic 201as discussed herein. For example, the image sensor and lens maytranslate laterally relative to each other in a direction perpendicularto an optical axis 205 of the camera 200. This in turn will laterallytranslate the field of view of the camera. For example, in FIG. 2A theimage sensor 204 is shown in a first position, and has a first field ofview 206 at the first position. If the image sensor 204 is movedlaterally to a second position 208 (shown in phantom), the camera willhave a second field of view 210 (shown in phantom) that is laterallytranslated relative to the first field of view 210. The optical axis 205of the camera 200 is shown in FIG. 2A as being positioned along a Z-axisof a cartesian coordinate system, and lateral translation in theseinstances occur along the X-axis and/or Y-axis (e.g., the first positionand second position of the image sensor 204 are laterally translatedalong the X-axis in FIG. 2A).

FIGS. 2B and 2C shows an example of a pair of images captured by thecamera 200 using laterally-shifted fields of view. Specifically, FIG. 2Bshows a first image 212 of a scene captured using the first field ofview 206, and shows a tree 214 toward the left side of the image and aperson 216 toward a right side of the image. FIG. 2C shows a secondimage 218 of the scene captured using the second field of view 210, inwhich the tree 214 has almost completely moved out of the field of viewand the person 216 is centered in the image. By laterally shifting aposition of the field of view of the camera 200, the camera 200 canselectively capture different content from a given scene, as discussedin more detail below.

When a camera of the devices described herein, such as camera 200, isconfigured to laterally move the field of view of the camera, thislateral movement may be used to perform optical image stabilization.When a camera performs optical image stabilization, the camera adjusts alateral position of the field of view (i.e., by relative movementbetween the lens and image sensor in a direction perpendicular to theoptical axis of the camera) in response to device movement, with theintent of maintaining the field of view of the camera to image aspecific portion of the scene. For example, when a user is holding adevice and capturing images using the camera, optical imagestabilization may compensate for movement of the camera resulting fromuser handshake, vibration, or the like. As a result, the camera maycapture sharper still images and videos with improved stabilization.

While optical image stabilization helps to maintain the field of view ona given portion of the scene in response to camera motion, thesetechniques do not impact the position of the field of view when thecamera is stationary. Conversely, the devices, systems, and methodsdescribed herein are directed to setting a default field of view. Asused herein, the “default field of view” of a camera is the size (whichmay be a fixed value in instances the camera has a fixed focal length)and lateral position of the camera's field of view when the camera isnot moving. Accordingly, when the camera is stationary, the camera maymaintain its field of view at the default field of view (i.e., at thesize and lateral position of the default field of view). In someinstances, the cameras described herein may also perform optical imagestabilization, during which the field of view will temporarily move fromthe default field of view in response to camera movement.

The devices described here may comprise a position control logic 201that is programmed to select a default field of view for a camera of thedevice. When setting the default field of view for a camera, theposition control logic 201 selects a lateral position of the defaultfield of view. In instances where the default field of view has avariable size (e.g., when the camera 200 is capable of optical zoom asdiscussed below), the position logic 201 may also set the size of thedefault field of view. In instances where the device includes multiplecameras with laterally moveable fields of view, there may be a singleposition control logic that selects default fields of view for multipleof these cameras. In other instances, there may be multiple positioncontrol logics that each set the default field of view for acorresponding camera. For example, a first position control logic mayselect the default field of view for a first camera, and a secondposition control logic may select the default field of view for a secondcamera.

The position control logic 201 may select the default field of viewusing one or more inputs. In some instances, an input includes positioninformation of a target object relative to the camera. Additionally oralternatively, an input includes orientation information associated withthe camera. These inputs will be described in more detail below. Whenthe position control logic 201 uses position and/or orientationinformation as described below, the position control logic 201 mayderive this information from information received by one or morecameras, depth sensors, other sensors (such as an accelerometer,directional sensor, gyroscope, and/or motion sensor as discussed above),or the like, or may receive this information after it has already beencalculated (e.g., by one or more processors elsewhere in the device orsystem).

When the position control logic 201 calculates the default field of viewfor a given camera, it may also set the instantaneous position and/orsize of the field of view of the camera using the calculated defaultfield of view. The position control logic 201 may drive actuation of thecamera to provide the necessary internal movement (e.g., movement of thelens 202 and/or image sensor 204) to move the field of view accordingly.When, for example, the camera is stationary, the position control logic201 may set the instantaneous position (and size) of the field of viewas the default field of view. In instances where the camera isconfigured to provide optical image stabilization, the position controllogic 201 may set the instantaneous position at a different positionfrom the default field of view while the camera is moving, where thesedifferent positions are calculated based on the motion of the camera.

In some instances, the camera 200 may be configured to create relativemovement between the image sensor 204 and the lens 202 in additionaldirections to provide additional functionality to the camera. FIG. 2Dshows some ways in which the camera 200 of FIG. 2A may facilitateadditional movement between the image sensor 204 and the lens 202. Forexample, an actuator of the camera 200 may be configured to createrelative movement between the image sensor 204 and one or more lenselements of the lens 202 along the optical axis 205 of the camera 200(as indicated by arrow 220). In some instances, this relative movementadjusts the focal plan of the camera 200, which actively changes thefocus of the camera. This may allow the camera 200 to provide autofocuscapabilities in which the camera automatically adjusts the focus of thecamera.

Additionally or alternatively, relative movement of one or more lenselements of the lens 202 along the optical axis may be used to changethe focal length of the lens 202, and thereby provide optical zoomfunctionality to the camera 200. In general, the size of the field ofview for a camera is inversely related to the focal length of thecamera. In other words, as the length of the focal length of the lens202 increases (i.e., the camera “zooms in”), the camera's field of viewwill narrow. Accordingly, it may be possible for the camera 200 to alterits field of view both via optical zoom (which will increase or decreasethe size of the field of view) and lateral shifting (which will change acenter of the field of view relative to the camera). At a given opticalzoom level, laterally shifting the field of view will maintain that samesize field of view while moving the field of view to capture a differentportion of a scene. It should be appreciated that there may be instanceswhere a camera simultaneously changes the size and lateral position ofthe field of view. For the purpose of this application, changing theoptical zoom of a camera without changing a center of the field of viewdoes not count as lateral movement of the field of view.

In some instances, the image sensor 204 may be tilted relative to thelens 202 (or vice versa). In these instances, one of the image sensor204 or lens 202 may be rotated around a direction perpendicular to theoptical axis 205 of the camera. While an imaging surface of the imagesensor 204 is shown in FIG. 2A is positioned perpendicular to theoptical axis 205 of the camera, the image sensor 204 is shown in FIG. 2Das being rotated around the Y-axis (as indicated by arrow 224) to tiltthe image sensor 204. This tilting changes the angle of the focal planeof the camera. When the image sensor 204 is positioned as shown in FIG.2A, the focal plane is perpendicular relative to the camera and thusobjects at a given distance from the camera will be in focus across theentire scene. When the image sensor 204 is tilted as shown in FIG. 2D,the focal plane will be at a non-perpendicular angle relative to thecamera, and the distance at which objects will be in focus will varyacross the scene. In this way, tilting the image sensor 204 impactswhich portions of a scene imaged by the camera 200 will be in focus.

Additionally or alternatively, the image sensor 204 may be configured torotate around the optical axis 205 of the camera, as indicated by arrow226 in FIG. 2D. In these instances, rotating the image sensor 204 aroundthe optical axis 205 will rotate the camera's field of view. While thisrotation may not change the center or size of the field of view, it maystill impact which portion of the scene is captured by the camera. Thisrotation may change the orientation of images captured by the camera.

FIGS. 2E and 2F show an example of a pair of images captured by thecamera 200 using rotated fields of view. Specifically, FIG. 2E shows animage 230 of a scene captured using an initial field of view, and showsa person 232. In this field of view, the image sensor 204 may be angledrelative to the ground, which causes the person 232 to appear tilted toone side. FIG. 2F shows another image 234 of the scene captured using afield of view that is rotated relative to the initial field of view,causing camera 200 to properly capture the orientation of the person 232as they stand in the scene.

In some instances, rotating the field of view of the camera 200 may beused to account for manufacturing misalignments between a camera 200 andthe device (e.g., device 100 discussed above) into which the camera 200is integrated, which cause the field of view of the camera to beslightly rotated relative to what is desired. While this could becorrected for in software by cropping and rotating a portion of theimages captured by the camera 200, this reduces the amount of the imagesensor that can be used for generating images. In other instances, thismisalignment may be measured (e.g., during factory calibration) and theimage sensor may be rotated by a predetermined amount during operationof the camera 200 to account for some or all of this misalignment,thereby allowing a larger portion of the imaging area of the imagesensor 204 to be utilized. Additionally or alternatively, rotation ofthe image sensor may be used to account for the orientation of thedevice, as will be discussed in more detail below.

It should be appreciated that any of the relative movement between theimage sensor and the lens discussed above (e.g., relative movement alongthe optical axis, relative movement laterally relative to the opticalaxis, or tilting relative to the optical axis of the camera) may beachieved via movement of the lens and/or the image sensor, depending onthe design of the camera. Additionally, in some instances the lens mayinclude one or more lens elements that can change their shape (e.g., aliquid lens). For the purpose of this application, the change of shapeof these lenses is considered to be movement to the extent that theshape change achieves one or more of the functions described above(e.g., changing the focus of the camera, laterally translating the fieldof view, or the like). Accordingly, the cameras described herein mayhave flexibility in the architecture used to create the movementdescribed with respect to FIGS. 2A-2F.

For example, FIG. 3 shows an illustrative example of a camera 300 thatmay be used with the devices, systems, and methods described herein. Asshown, camera 300 includes a lens 302, an image sensor 320, a housing304, and an actuator that is configured to move the lens 302 along anoptical axis of the camera to adjust the focus of the camera and to movethe image sensor 320 transverse to the optical axis to laterally movethe field of view of the camera 300. To move the lens 302 (which maycontain one or more lens elements and a lens barrel, such as discussedabove), the lens 302 is attached to a coil 312 (e.g., via a lens carrier310, which is configured to hold both the lens 302 and the coil 312).The lens 302 and lens carrier 310 are suspended relative to a stationaryportion of the camera 300 (e.g., relative to a magnet holder 306) viaone or more suspension elements (not shown), such as one or moreflexures (e.g., leaf spring(s), suspension wire(s), flexure arms(s), orthe like) and/or one or more bearings (e.g., ball bearing(s), rollerbearing(s), or the like). The magnet holder 306 holds one or moremagnets 308, and the coil 312 is positioned within the magnetic field ofthe magnets 308 such that when current is driven through the coil 312,Lorentz forces are generated that can create relative movement betweenthe coil 312 and magnets 308, which in turn may move the lens 302 alongthe optical axis of the camera.

To move the image sensor 320 in one or more directions perpendicular tothe optical axis and laterally shift a position of the camera's field ofview, the camera 300 includes a stationary base 314 (which may be fixedrelative to housing 204) and a frame 322 that moveably connects theimage sensor 320 to the base 314 via a plurality of flexures 326. Insome instances the flexures 326 support electrical traces 324, which maybe used to carry signals to and from the image sensor 320. Also shownthere is a printed circuit 318 which carries one or more coils 316(which are separate from the coil 312 carried by the lens carrier 310).The coils 316 may be positioned within the magnetic field of the one ormore magnets (e.g., magnets 308 in the variation shown in FIG. 3 ) suchthat when current is driven through the coils 316, Lorentz forces aregenerated that can create relative movement between the coils 316 andmagnets 308, which in turn may move the image sensor 320 perpendicularto the optical axis of the camera. In some instances, the coils 316 andmagnets may be positioned and controlled to provide Lorentz forces in amanner that cause the frame 322 and image sensor 320 to rotate aroundthe optical axis of the camera.

In some instances, the camera 300 may be further configured to tilt thelens 302 and/or image sensor 320 relative to the optical axis of thecamera 300. For example, to tilt the lens 302 relative to the opticalaxis of the camera 300, the camera 300 may include multiple coilsattached to the lens (either replacing or in addition to coil 312). Inthis instance the coils may be positioned on different sides of thelens, and may interact with one or more magnets in the camera (e.g.,magnets 308) to create Lorentz forces as discussed above. These coilsmay be controlled to provide unequal Lorentz forces directed along theoptical axis, which may cause the lens 302 to rotate and thereby tiltrelative to the optical axis. If the coils are controlled to provideequal Lorentz forces, the lens 302 may move along optical axis of thecamera 300 without tilting. Similarly, the camera 300 may be configuredto tilt the image sensor 320 using one or more coils that provideunequal Lorentz forces directed along the optical axis. Additionalnon-limiting examples of cameras that move a lens and/or image sensor inmultiple directions are described in U.S. Patent Application PublicationNos. US 2021/0132327 (titled “Camera Actuator for Lens and SensorShifting”) and US2021/0080807 (titled “Camera Focus and StabilizationSystem”), the contents of which are incorporated herein by reference intheir entireties.

As mentioned above, the cameras described herein may be configured toadjust a lateral position of a default field of view for the camera.This allows the camera to visualize different portions of a given sceneas may be desired. For example, in some instances a lateral position ofa camera's field of view may be determined at least in part based on aposition of an object within a scene. For example, FIG. 4A shows anexample of a camera 400 that is used to capture an image stream of aperson 402 positioned in a scene surrounding the camera. The camera 400includes an image sensor 404 and a lens 406, and is configured to createrelative movement between the image sensor 404 and lens 406 to laterallyshift a field of view of the camera 400.

In FIG. 4A, the person 402 is positioned at a first location in thescene and the camera 400 has a first field of view 408 having a firstlateral position. For the purpose of this application, when a person orobject is discussed as being positioned at a location, this refers tothe location of a representative point (or multiple points) in a spacethat is selected to represent the person or object. For example, for aperson, the representative point may be selected to correspond to apredetermined portion of the person's face or body, such as a center ofthe face, a point between the eyes of the person, a point on their nose,or the like. For a non-person object, the representative point may be acenter of the object or another point that may depend on the type ofobject.

While the person 402 is in the first position, the camera 400 maycapture one or more images with the first field of view 408, which maybe used to generate one or more output images. These output images maybe used for a number of purposes. For example, the output images may beused to generate a live preview that will be displayed during a camerauser interface that displays the stream of output images. The outputimages illustrate to a user the portion of the field of view that willbe saved when the camera initiates a media capture event. When thecamera initiates a media capture event (under some predeterminedconditions or when a user gives a command to capture images byinteracting with a shutter control on a user interface, pressing adesignated button on the device, giving a voice command, or the like),the camera will capture media depending on the current camera mode(e.g., capture a still image when in a photo mode or capture a videowhen in a video mode), which may then be stored locally on the device100 or transmitted to a remote server for storage.

It should be appreciated that while the camera 400 will capture imageshaving the full field of view of the camera, the output images may onlyrepresent a subset of the camera's field of view as mentioned above. Forexample, FIG. 4B shows an example image 410 captured by the camera 400using the first field of view 408. Also shown there is a croppingboundary 412 that represents the portion of the image 410 that will beused to generate a corresponding output image (e.g., that is displayedin a live preview or stored as part of an image or video). In theseinstances, although additional image information is captured by theimage sensor of the camera, the user will only see a cropped subset ofthe camera's field of view corresponding to the cropping boundary 412.

As an example, images captured by the camera 400 may be used tofacilitate a video conferencing session. For example, FIG. 4C shows adevice 414 that includes a camera 400 that may be used to facilitate avideo conferencing session. The device 414, which may be configured inany manner as described above with respect to FIGS. 1A-1C, is shown inFIG. 4C as having a display 416 that displays a communication userinterface 418. The communication user interface 418 includes a firstvideo feed 422 and a second video feed 424. The first video feed 422 isa representation of an image stream captured by the camera 400(including person 402), while the second video feed 424 is arepresentation of image data captured by a camera from a second device(not shown) that is communicated from the second device to device 414during the video conferencing session. The first video feed 422 may betransmitted from device 414 to the second device during the videoconferencing session to allow the first video feed 422 to be displayedfrom the second device. In the variation shown in FIG. 4C, a croppedportion of the camera's field of view corresponding to the croppingboundary 412 is used to generate the output images that form the firstvideo feed 422. Also shown in FIG. 4C is a set of optional controls 426that may be displayed in the communication user interface 418, which maybe used (e.g., via corresponding user inputs to a touch-sensitivesurface of the display) to control one or more aspects of thevideoconferencing session (e.g., muting audio, applying visual effectsto one of the video streams, terminating the videoconferencing session,or the like).

The camera 400 may change the lateral position of the field of viewresponsive to movement of the person 402 in the scene. For example, FIG.4D shows the camera 400 imaging the person 402 while the camera islocated in the same location as in FIG. 4A but the person 402 ispositioned at a second location in the scene that is different that theperson's first position from FIG. 4A. The image sensor 404 is laterallytranslated relative to the lens 406, thereby providing the camera with asecond field of view 428 having a second lateral position different thanthe first lateral position. FIG. 4E shows an example image 430 capturedby the camera 400 using the second field of view 428. As shown there,the person 402 is still centered in the image 430 even though therelative positioning between the person 402 and camera 400 has changed.If less than the full field of view of the camera is used (such as withthe first video feed 422 in FIG. 4C), the cropping boundary 412 may keepits relative position with the field of view, and images generated fromthe cropping boundary 412 will also continue to include the person 402.

In some instances, it may also be desirable to set a relative tiltbetween the image sensor 404 and lens 406 (as discussed above withrespect to FIG. 2D) based on the relative position between the person402 and the camera 400. Depending on where the person 402 is located inthe scene and how they are orientated relative to the camera, it may bepossible that part of the person 402 is out of focus. For example, whenthe user is positioned as shown in FIG. 4D, a region 432 of the user isout of focus. As shown in FIG. 4F, the image sensor 404 may be tiltedrelative to the lens 406 (or vice versa), thereby changing the angle ofthe focal plane of the camera. The amount of tilt may be selected suchthat the region 432 is again in focus, as illustrated in FIG. 4G by animage 434 of the person 402 captured while the components of camera 400are positioned as shown in FIG. 4F.

While the first field of view 408 shown in FIG. 4A and the second fieldof view 428 shown in FIG. 4D are positioned such that the person 402stays centered in the camera's field of view, in some instances it maybe possible for the person to be imaged at different positions within agiven field of view. For example, when a cropping boundary 412 is usedto select a subset of the field of view for generating an output image,it may be desirable to maintain the person 402 at a predeterminedposition within the cropping boundary 412. This allows the person 402 tomaintain their position within output images generated from image datain the cropping boundary 412. There may be instances, however, where therelative position of the person 402 within the cropping boundary 412 maybe maintained without needing to move the camera's field of view.

For example, FIG. 4H shows the camera 400 imaging the person 402 whilethe camera is located in the same location as in FIG. 4D but the person402 is positioned at a third location in the scene that is differentthat the person's second position from FIG. 4D. In this instance, thecamera 400 is still configured to image the second field of view 428having the second lateral position, and the person 402 is stillpositioned within the second field of view 428. Accordingly, an image436 captured by the camera 400 will still capture the person 402, asshown in FIG. 4I, but the person 402 is no longer centered in thecamera's field of view 428. The relative position of the croppingboundary 412 within the camera's field of view may instead be moved inorder to maintain the position of the person 402 within the croppingboundary 412. In this way, the camera may still generate output imagesthat contain the person 402. By utilizing both a moving croppingboundary 412 and a laterally shifting field of view, the camera 400 cangenerate output images that include the person 402 across a wider extentof the scene than using either technique alone. Additionally, this canall be done without requiring the user to reposition the camera 400.

A moveable cropping boundary 412 may be used with a laterally shiftingfield of view in a number of ways. In some instances, the camera 400 maytry to keep the person 402 at or near the center of the field of viewwhen possible. Depending on the configuration of the camera 400, theimage quality may be better near the center of the field of view due tovignetting or image distortions that may occur at the periphery of thefield of view. Accordingly, the field of view of the camera 400 may bedivided into a central region and a peripheral region at least partiallysurrounding the central region, and the camera 400 will attempt toselect a default field of view having a lateral position at which theperson 402 is positioned in the central region. In other words, theposition control logic of a camera may determine whether a set ofcandidate positions exists that would place the person 402 in the centerregion. Upon a determination that candidate positions exist, theposition control logic will select one of the candidate positions as thelateral position of the default field of view.

If the camera 400 is unable to move the field of view enough such thatthe person is positioned in the central region, the camera 400 willselect a default field of view having a lateral position at which theperson 402 is positioned in the peripheral region. In other words, upondetermining that there are no candidate positions that would place theperson in the center region, the position selection logic may insteadselect a lateral position that positions the object in the peripheralregion. The cropping boundary 412 may be moved as needed to continuegenerating output images that include the person.

In some examples, the camera 400 will attempt to position the person 402at a predetermined position within the field of view. In theseinstances, the central region is limited to a single target point, suchas the center of the field of view, and the camera 400 will attempt toselect a default field of view that positions the person 402 at thistarget point. The cropping boundary 412 may maintain its relativeposition while camera 400 is able to keep the person 402 at the targetpoint, but then may change its position within the field of view as thecamera 400 is no longer able to keep the person 402 at the target point.

In other examples, there may be multiple possible positions for thefield of view that would position the person 402 in the central region,and the camera 400 will select a default field of view that minimizespower consumption by the camera 400. Specifically, the field of view ofthe camera 400 may have a resting position, which represents theposition of the field of view when the camera 400 is not activelychanging the field of view of the camera 400. Depending on the design ofthe camera 400, this resting position may be dependent on an orientationof the camera 400 due to the relative direction of gravitational forceson components of the camera 400 in different orientations. The camera400 consumes power to move the field of view and/or hold the field ofview at a position different than the resting position, and the amountof power consumed may depend on how far the current position of thefield of view is from the resting position. Accordingly, the camera 400can save power by keeping the field of view at or near the restingposition when possible.

For example, the camera 400 may first attempt to set the restingposition as the default field of view if doing so would position theperson 402 within the central region. If the camera is able to do this,it may maintain the resting position as the lateral position for thedefault field of view for as long as the person 402 remains located inthe central region. As the person 402 moves to different locations thatare still within the central region, the cropping boundary 412 may moveits relative location to track the person 402 without needing to changethe lateral position of the default field of view (and thereby consumeadditional power). If the person 402 moves to a location outside thecentral region, the camera 400 may select a new default field of viewthat keeps the person 402 in the central region, if possible. Again,this new default field of view may be selected at a lateral positionthat consumes the least amount of power while still maintaining theperson in the central region.

While the example of camera 400 is discussed above as setting a defaultposition based on the location of a person 402 within a scene, it shouldbe appreciated that these techniques may be used to adjust the field ofview of a camera FIG. 5 shows a flowchart that represents an examplemethod 500 that a position control logic may utilize to select thedefault field of view for a camera of a device. Specifically, theposition control logic may be configured to obtain position informationassociated with a target object at step 502. The target object may be aperson or a non-person object, and may be selected using any suitablemanner (e.g., selected based on object detection within images capturedby the camera, selected based on a user input, or the like). In someinstances multiple objects may be selected, and the target object may bea region of interest (“ROI”) that encompasses two or more objects (itshould be appreciated that the ROI may include two sections that neednot be contiguous) or may be an ROI that corresponds to a single object(which may be selected from the multiple objects based on one or morepredetermined criteria).

The position information includes a lateral location of the object(i.e., where the object falls within or relative to the field of view).In some instances, the position information also includes a proximityvalue representing a distance between the object and the camera.Additionally or alternatively, the position information may includerelative orientation information that represents a relative orientationbetween the object and the camera (e.g., whether a person is facing thecamera or positioned at an angle relative to the camera). This positioninformation may be obtained using any suitable technique, such asperforming image analysis of images captured by the camera (or byanother camera or cameras of the device), analysis of depth informationcaptured by a depth sensor of the device, combinations thereof, or thelike.

At step 504, the position control logic is configured to set a lateralposition of the default field of view of the camera using the positioninformation of the object. If the lateral position of the default fieldof view set at step 504 is different than the current default field ofview, the camera will laterally shift the field of view of the camera(e.g., via relative movement of a lens and image sensor of the camera asdiscussed previously) to the new lateral position. The positioninformation of the object may be used in any manner as described abovewith respect to FIGS. 4A-4I to select the lateral position of thedefault field of view. For example, the position information of theobject may be used to position the object within a target region of thefield of view (e.g., a central region discussed previously) if possible.

In some instances, the position control logic is optionally configuredto set a size of the default field of view using the positioninformation of the object at step 506. If the camera has optical zoomcapabilities, the focal length of the camera may be updated to changethe size of the default field of view. The position control logic may beconfigured to change the size of the default field of view based on theproximity of the object to the camera, such as increasing the size ofthe default field of view as an object approaches the camera. In otherinstances, the position control logic may be configured to change thesize of the default field of view based on the location of the object.For example, if the object has moved toward an edge of the default fieldof view but the camera is unable to further shift the field of view inthis direction, the camera may zoom out to increase the size of thefield of view. This may allow the camera to image the object over aneven wider range of scene positions. Additionally or alternatively, theposition control logic may also optionally set a relative tilt betweenthe image sensor and the lens using the position information of theobject at step 508. This allows the camera to adjust the focal plane ofthe camera as discussed above.

Once the default field of view (and relative tilt in instances where themethod performs step 508) has been updated (and the camera has performedany actuation necessary to move the field of view to the selecteddefault field of view), the camera captures an image at step 510. Thecamera may continue capturing images until one or more of the inputs tothe position control logic are updated (at step 512) at which point anew iteration of the method 500 may be started again at step 502. Theseupdates may occur after each image capture, or may be performedperiodically such that multiple images are captured by the camera beforethe inputs to the position control logic are updated. Accordingly thecamera may capture an image stream with a default field of view thatdynamically updates as the object moves.

This captured image stream may be used (either by the device thatincludes the camera or another device that receives the captured imagestream therefrom) to generate one or more output images as discussed inmore detail above. In some instances, a cropping boundary may be appliedto the captured image stream to set a boundary of the output images. Thesize and position of the cropping boundary applied to a captured imagemay be set using the position information for the object. For example,the relative position of the cropping boundary within the image streammay move to account for movements of the object within the field ofview, such as described above with respect to FIGS. 4H and 4I.

As an example, the output images generated from the captured imagestream may form the first video feed 422 in the videoconferencingsession described above with respect to FIG. 4C. As the person 402 movesrelative to the device 414, the default field of view (and the croppingboundary in instances where only a portion of the captured images areused to generate the first video feed) may adjust to maintain the person402 at a target position within the video feed 422. Accordingly, theperson 402 may be captured across a wide range of scene locationswithout requiring a user to manually change the position of the device414.

In other instances, it may be desirable to use the camera of a device togenerate output images directed to a surface (e.g., a surface of atable, counter, desk or the like) or an object positioned on a surfacethat is identified in a scene surrounding the camera. For example, FIG.6A shows a system 600 that includes a device 602 with a camera 604 (notshown in FIG. 6A) having a field of view 606. The device 602 may bepositioned relative to a surface 608, such that a portion of the surface608 is positioned within the field of view 606 of the camera 604. FIG.6B shows an example image 610 captured by the camera 604 when the device602 is positioned as depicted n FIG. 6A. A portion of the surface 608that is captured in the image 610 may be used in generating an outputimage as discussed above.

For example, in the variation shown in FIGS. 6A and 6B, a piece of paper612 may be positioned on the surface 608, and a cropping boundary 614 isused to select a portion of the surface 608 that includes the piece ofpaper 612 for use in generating output images. These output images maybe used to facilitate a video conferencing session as illustrated inFIG. 6C. Specifically, 6C shows a front view of the device 602, which inthis variation includes camera 604 and a display 616 that displays acommunication user interface 618. The communication user interface 618includes a first video feed 622 and a second video feed 624. The firstvideo feed 622 is a representation of an image stream captured by thecamera 604, while the second video feed 624 is a representation of imagedata captured by a camera from a second device (not shown) that iscommunicated from the second device to device 414 during the videoconferencing session. The first video feed 422 may be transmitted fromdevice 414 to the second device during the video conferencing session toallow the first video feed 422 to be displayed from the second device.

A cropped portion of the camera's field of view corresponding to thecropping boundary 614 is used to generate the output images that formthe first video feed 622, which includes the piece of paper 612 in thevariation shown in FIG. 6C. In some instances the cropping boundary 614may have a non-rectangular shape (though a rectangular shape may be usedif so desired), and the image data associated with the cropping boundary614 may be modified to generate an output image having a rectangularshape (e.g., to correct for perspective distortions resulting from therelative position between the camera 604 and the surface 608). Alsoshown in FIG. 6C is a set of optional controls 626 that may be displayedin the communication user interface 618, which may be used (e.g., viacorresponding user inputs to a touch-sensitive surface of the display)to control one or more aspects of the videoconferencing session such asdiscussed previously.

Generating the first video feed 622 from a portion of a surface 608 maybe one example of a number of different imaging modes that a user mayselect during a videoconferencing session. For example, the system 600may detect a user input (e.g., via an input to one of the controls 626,a keyboard input, a gesture, an audio input, or the like) to change thevideo conferencing session to a surface-focus mode. While the videoconferencing session is in the surface-focus mode, the first video feedis generated using captured images (or a cropped portion thereof) thatinclude a portion of the surface 608.

The ability of the camera 604 to image the surface 608 depends at leastin part on the relative orientation and positioning between the camera604 and the surface 608. For example, in some instances the system 600further comprises an accessory device 630 that may hold the device 602and camera 604 in a particular orientation. The accessory device 630 maybe positioned in a fixed relationship relative to the surface 608, andmay thereby hold the device 602 and camera 604 in a particularorientation relative to the surface 608. For example, in the variationshown in FIG. 6A, a portion of the accessory device 630 may be placeddirectly on the surface 608 to set a relative position between theaccessory device 630 and the surface 608.

The accessory device 630 may be any device capable of holding the device602 and camera 604 in a particular orientation (or one of a number ofpossible orientations). In some examples, the accessory device 630includes a foldable cover that can be folded into a support structure(e.g., having a triangular shape). The support structure, when placed ona surface, may hold the device 602 at a predetermined angle relative tothat surface. In other instances, a laptop or computer monitor may actas the accessory device 630. As an example, the device 602 may betemporarily attached to an upper housing portion of a laptop (e.g., theportion that supports a display of the laptop), and a lower housingportion of the laptop (which may include a touchpad, keyboard, or thelike and is connected to the upper housing portion via a hinge) may beplaced on a surface. The angle between the upper housing portion and thelower housing portion may set the relative orientation and positioningbetween the device 602 and the surface 608, and thereby set the relativeorientation and positioning between the camera 604 and the surface 608.

In the variation shown in FIG. 6A, the accessory device 630 isconfigured as a folio that may be adjusted to hold the device 602 in anyof multiple different positions and orientations relative to a surface.In this variation, the accessory device 630 includes a base segment 632that may be on the surface 608 to set the relative position between theaccessory device 630 and the surface 608. The base segment 632, which insome instances may include a keyboard, touchpad, or the like, isrotatably coupled to a second segment 634 via a first hinge assembly636. The second segment 634 is rotatably coupled to a third segment 638via a second hinge assembly 640. The device 602 may be temporarilysecured to the third segment 638 (e.g., via magnets, mechanicalfasteners, or the like). The second segment 634 and third segment 638suspend the device 602 relative to the base segment 632 and the surface608. Specifically, the angle between the base segment 632 and the secondsegment 634 controls the distance between the device 602 and the surface608, while the angle between the second segment 634 and the thirdsegment 638 controls the angle of a front surface the device 602relative to the surface 608. In this way, a user may adjust either orboth of the height and angle of the device 602 relative to the surface608.

For example, in FIG. 6A the system 600 is shown such that the device 602is positioned with a front surface of the device 602 positionedperpendicular to surface 608. FIG. 6D shows the same system as FIG. 6A,except that the third segment 638 of the accessory device 630 has beenrotated to change the relative angle between the device 602 and thesurface 608. Specifically, the front surface of the device 602 has beenrotated to face away from the surface 608. If the field of view of thecamera 604 of the device 602 were to keep the same lateral position 646(shown in phantom in FIG. 6D) as the default field of view 606 depictedin FIG. 6A, the surface 608 and piece of paper 612 would no longer bepositioned in the camera's field of view (and thus wouldn't be presentin images captured by the camera 604). In these instances, the field ofview of camera 604 may be moved to a second lateral position 648 asdiscussed above to maintain the surface 608 and piece of paper 612 inthe camera's field of view. In this way, even though the device 602 andcamera 604 have been rotated away from the surface 608, the camera 604may still capture images similar to image 610 of FIG. 6B.

Accordingly, the systems, devices, and methods described herein may beused to image a portion of a surface. In some instances, an object on asurface (such as the piece of paper 612 described previously) may beselected as a target object, and the object-based techniques describedabove with respect to FIGS. 4A-5 may be used to adjust the default fieldof view based on a position of the target object. This may facilitategenerating output images that include the target object, even as a usermoves the target object relative to the surface.

In other instances, it may be desirable to set a lateral position of thedefault field of view based at least in part on the orientation of thecamera. This may allow the device to image a fixed region of space(e.g., a particular portion of a surface) over a range of possibledevice orientations. For example, during a video conferencing session asdiscussed above, a user may wish to switch from an object-focus mode(e.g., when the field of view is adjusted based on a position of anobject within the field of view, such as discussed above with respect toFIGS. 4A-5 ) to a surface-focus mode to image a portion of a surface asdiscussed above. In these instances, using orientation informationassociated with the camera to set a lateral position of the defaultfield of view may allow the camera to image the surface across a rangeof orientations. This gives a user the flexibility to switch betweenthese videoconferencing modes without needing to reorient or repositionthe camera.

FIG. 7 shows a flowchart that represents an example method 700 that aposition control logic may utilize to select the default field of viewfor a camera of a device. Specifically, the position control logic maybe configured to obtain orientation information associated with a cameraat step 702. This orientation information can include absoluteorientation information (i.e., orientation of the camera relative togravity) or a relative orientation information (e.g., orientation of thecamera relative to another object), and can be measured in any suitablemanner.

For example, when the orientation information includes absoluteorientation information, the orientation of the camera relative togravity may be calculated using one or more sensors such asaccelerometers, gyroscopes, combinations thereof, or the like. In someinstances, the camera may include one or more sensors that provideabsolute orientation information. Additionally or alternatively, adevice incorporating the camera may include one or more sensors thatprovide absolute orientation information of the device. Because therelative position and orientation of the camera within a device isknown, orientation information of the device may be used to calculate(or otherwise used in place of) orientation information of the camera.

In some instances, relative orientation information may include therelative positioning of the camera relative to another object. This mayinclude a relative angle between the camera and the object (e.g., anangle between the optical axis of the camera and a target surface of theobject), and in some instances a distance between the camera and theobject. In some of these variations, the relative orientationinformation includes the relative orientation between the camera and atarget surface. For example, depth information from a depth sensor orimages from a camera may be analyzed to detect the relative angle and/ordistance between the camera and the plane.

Additionally or alternatively, when the device incorporating the camerais coupled (e.g., releasably coupled) to an accessory device, therelative orientation information includes the relative orientationbetween the camera and a target portion of the accessory device. In someinstances, the device incorporating the camera is configured tocommunicate with the accessory device (e.g., via an electrical connectoror the like), and receives information about a current configuration ofthe accessory device therefrom. In some of these variations, theaccessory device may include one or more sensors (such as anaccelerometer, directional sensor, gyroscope, and/or motion sensor asdiscussed above), that may be used to determine the orientation of theaccessory device (or relative orientation of components of the accessorydevice). For example, in the example of the accessory device 630 shownin FIGS. 6A and 6D, the accessory device 630 may be able to communicatethe relative positions of the base segment 632, second segment 635, andthird segment 638 to the device 602, which may be used to calculate therelative angle and distance between the camera 604 and the base portion632. As another example, in an instance where the accessory device is alaptop or monitor, the accessory may include one or more sensors thatcan determine a relative angle between a screen (or a portion of theaccessory device housing the screen) and a base of the accessory device.

Additionally or alternatively, depth information from a depth sensor orimages from a camera may be analyzed to detect a portion of theaccessory device, and may use that information to determine relativeorientation information between the camera and the accessory device. Forexample, when the device 602 described with respect to FIGS. 6A-6D ispositioned as shown in FIG. 6A, some of the base portion 632 ispositioned within the field of view 606 of the camera 604. By analyzingthe amount and shape (e.g., how much distortion is present) of the baseportion 632 is present in images captured by the camera 604, theposition control logic may determine a relative angle and/or distancebetween the camera and the base portion 632.

At step 706, the position control logic is configured to set a lateralposition of the default field of view of the camera using theorientation information associated with the camera. If the lateralposition of the default field of view set at step 706 is different thanthe current default field of view, the camera will laterally shift thefield of view of the camera (e.g., via relative movement of a lens andimage sensor of the camera as discussed previously) to the new lateralposition. Any of the orientation information described above may be usedto set the default field of view of the camera.

For example, in some instances the lateral position of the default fieldof view of the camera is set using absolute position information of thecamera. In these instances, the lateral position of the default field ofview may vary as a function of an angle of the camera (e.g., the opticalaxis of the camera) relative to gravity. If the camera rotates to changeorientations, the default field of view may be shifted in a directionopposite of the direction of rotation. This may allow a user to placethe camera in any of a number of different orientations, and the defaultfield of view will be selected so that the same portion of a scene is inthe camera's field of view regardless of the camera's orientation.

In other instances, the lateral position of the default field of view ofthe camera is set using relative orientation information between thecamera and a surface that is present in the scene around the camera.This may include setting the position based on a relative angle and/ordistance between the camera and the surface. As the camera is angledaway from the surface, the default field of view may be shifted towardthe surface (and vice versa) in order to keep a target portion of thesurface in the camera's view. This may provide a consistent area on asurface in which a user places documents or other objects to be imagedby the camera. Using this relative orientation information may beadvantageous in situations where the camera is being used on a non-levelsurface.

In still other instances, the lateral position of the default field ofview of the camera is set using relative orientation information betweenthe camera and a target portion of an accessory device. For example, theaccessory device may have a portion thereof that falls in the camera'sfield of view for certain camera orientations (e.g., base portion 632 ofthe accessory device 630 of FIG. 6A). In these instances, it may bedesirable to set the default field of view such that the accessorydevice is not positioned in the field of view (or such that apredetermined amount of the accessory device is present in the field ofview). This may reduce the amount of the camera's field of view taken upby the accessory device, thereby allowing more of the field of view tobe dedicated to imaging other portions of the scene.

It should be appreciated that the lateral position of the default fieldof view may be set using multiple types of orientation information(e.g., different combinations of absolute and relative orientationinformation as discussed above), and that the position control logic mayutilize a different type of orientation information at different times.Additionally, in some instances the position control logic may beconfigured to set a lateral position of the default information usingboth orientation information associated with the camera and positioninformation of a target object.

For example, during a video conferencing session, instead of selectingeither an object-focus mode or a surface-focus mode, a user may opt fora hybrid-focus mode in which a device captures images that include botha target object and a target surface. FIG. 8A shows an example of asystem 800 that may facilitate a hybrid-focus mode. The system 800includes a device 802 that includes a camera 804 (not shown in FIG. 8A)having an adjustable field of view 806 as discussed above. The device802 may be positioned relative to a surface 808 (e.g., using anaccessory device 810 such as those discussed above), such that a portionof the surface 808 is positioned within the field of view 806 of thecamera 804. In these instances, a person 812 may also be positioned inthe field of view 806 of the camera 804. Accordingly, images captured bythe camera 804 may include both the person 812 and the surface 808, asillustrated in FIG. 8B by an example image 814 captured by the camera804 when the device 802 is positioned as shown in FIG. 8A.

These images may be used to generate one or more video streams. Forexample, in the variation shown in FIGS. 8A and 8B, a piece of paper 816may be positioned on the surface 808, and a first cropping boundary 818is used to select a portion of the surface 808 that includes the pieceof paper 816 for use in generating a first set of output images. Asecond cropping boundary 820 is used to select another portion of thecamera's field of view associated with the person 812 for use ingenerating a second set of output images. These sets of output imagesmay be used to facilitate a video conferencing session as illustrated inFIG. 8C. Specifically, FIG. 8C shows a front view of the device 802,which in this variation includes the camera 804 and a display 822 thatdisplays a communication user interface 824.

The communication user interface 824 includes a first video feed 828, asecond video feed 830, and a third video feed 832. The first and secondvideo feeds 828 and 830 are representations of an image stream capturedby the camera 804. Specifically, the first video feed 828 corresponds tothe first bounding box 818, and includes output images that aregenerated to include the portion of the surface 808 including the pieceof paper 816. The second video feed 830 corresponds to the secondbounding box 820, and includes output images that are generated toinclude the person 812. The third video feed 832 is a representation ofimage data captured by a camera from a second device (not shown) that iscommunicated from the second device to device 802 during the videoconferencing session. The first and second video feeds 828 and 830 maybe transmitted from device 802 to the second device during the videoconferencing session to allow the first and second video feeds 828 and830 to be displayed from the second device.

Returning to FIG. 7 , in these instances the method may optionallyinclude obtaining position information for a target object at step 704.The selection of a target object and determination of positioninformation of the target object may be performed in any manner asdescribed above with respect to the method 500 of FIG. 5 . In theseembodiments, setting the lateral position of the default field of viewusing orientation information associated with the camera at step 706also includes setting the lateral position of the default field of viewusing the position information for the target object.

In some of these variations, the position control logic may determinepotential lateral positions for the default field of view using theorientation information associated with the camera, and the positioncontrol logic may use the object position information to select one ofthese potential lateral positions and set the default field of view tothis selected lateral position. This may allow the field of view to moveto accommodate movement of an object (such as the person 812 in FIGS.8A-8C) using the techniques discussed above with respect to FIGS. 4A-5 ,but the system will constrain this movement to a set of positions thatwill not interfere with capturing another portion of a scene (such asthe piece of paper 816 in FIGS. 8A-8C).

This may be useful in instances where the image stream is used for ahybrid-focus mode in a video conferencing session as discussed above, aswell as in instances where a user is switching between modes. Forexample, a user may switch between an object-focus mode (during whichthe position control logic may set a lateral position of the defaultfield of view using position information for a target object), asurface-focus mode (during which the position control logic may set thelateral position based on camera orientation information), and ahybrid-focus mode (during which the control logic may set the lateralposition based on both camera orientation information and positioninformation for a target object).

In some instances, the position control logic is optionally configuredto set a size of the default field of view at step 708. The size of thedefault field of view may be set using the camera orientationinformation, using the position information of the target object (to theextent that information is obtained at step 704), combinations thereof,or the like. If the camera has optical zoom capabilities, the focallength of the camera may be updated to change the size of the defaultfield of view. The position control logic may be configured to changethe size of the default field of view based on the proximity of thecamera to an object or a surface, such as increasing the size of thedefault field of view as the distance between the camera and the objector surface decreases. In other instances, the position control logic maybe configured to change the size of the default field of view based onthe location of a target object, as discussed above.

Additionally or alternatively, the position control logic may alsooptionally set a rotation of the field of view using the cameraorientation information at step 710. For example, if a device (or anaccessory device holding the device) is not placed flat on a surface(e.g., one side of the device is propped up on another object to anglethe device relative to the surface), images captured by the camera maybe rotated as such as shown in FIG. 2E. In these instances, theorientation information of the camera may be used to identify that thecamera is angled relative to the surface, and the image sensor may berotated around the camera's optical axis to correct for thismisalignment.

Additionally or alternatively, the position control logic may alsooptionally set a relative tilt between the image sensor and the lens atstep 712. This allows the camera to adjust the focal plane of the cameraas discussed above. This tilt may be set using the camera orientationinformation, using the position information of the target object (to theextent that information is obtained at step 704), combinations thereof,or the like. In some instances, the tilt may be based on user input toselectively adjust the focus of a portion of the scene. For example,FIGS. 9A and 9B show a first image 900 and a second image 910,respectively, captured by a camera that can create a relative tiltbetween the image sensor and the lens (such as camera 200 discussedabove). Depending on the design of the camera and the relative positionand dimensions of the objects being imaged, an object such as a piece ofpaper 902 may be positioned in the camera's field of view such only aportion of the object can be in focus at a given time. For example, inFIG. 9A, a first portion 904 of the piece of paper 902 may be out offocus (as illustrated with hatching) while a second portion 906 of thepiece of paper 902 is in focus.

A user may indicate a desire to change the focus to bring the firstportion in focus by providing an input to the system. For example, inFIG. 9A a stylus 908 or other object (such as a user's finger) may beplaced in the camera's field of view, and a location of the stylus 908may be used to indicate a portion of the piece of paper 902 that theuser wants in focus. Alternatively, a user may select a portion of alive preview or video stream via corresponding user inputs to atouch-sensitive surface of the display to select this portion of thepaper 902. The system may then adjust the relative tilt between the lensand image sensor of the camera to place the first portion 904 of thepiece of paper 902 in focus, as depicted in FIG. 9B.

Once the default field of view (and relative tilt in instances where themethod performs step 712) has been updated (and the camera has performedany actuation necessary to move the field of view to the selecteddefault field of view), the camera captures an image at step 714. Thecamera may continue capturing images until one or more of the inputs tothe position control logic are updated (at step 716) at which point anew iteration of the method 700 may be started again at step 700. Theseupdates may occur after each image capture, or may be performedperiodically such that multiple images are captured by the camera beforethe inputs to the position control logic are updated. Accordingly thecamera may capture an image stream with a default field of view thatdynamically updates as the object moves.

This captured image stream may be used (either by the device thatincludes the camera or another device that receives the captured imagestream therefrom) to generate one or more output images as discussed inmore detail above. In some instances, one or more cropping boundariesmay be applied to the captured image stream to set a boundary of one ormore sets of output images, such as described above with respect toFIGS. 8A-8C.

The foregoing description, for purposes of explanation, uses specificnomenclature to provide a thorough understanding of the describedembodiments. However, it will be apparent to one skilled in the art,after reading this description, that the specific details are notrequired in order to practice the described embodiments. Thus, theforegoing descriptions of the specific embodiments described herein arepresented for purposes of illustration and description. They are nottargeted to be exhaustive or to limit the embodiments to the preciseforms disclosed. It will be apparent to one of ordinary skill in theart, after reading this description, that many modifications andvariations are possible in view of the above teachings.

What is claimed is:
 1. A system comprising: a device comprising acamera, the camera having an optical axis and an adjustable field ofview, wherein the camera comprises: a lens; an image sensor; and aposition control logic configured to: obtain camera orientationinformation associated with the camera; select a lateral position of adefault field of view using the camera orientation information; controla relative position of the lens and the image sensor to maintain theadjustable field of view as the default field of view while the camerais stationary; and capture an image at the default field of view.
 2. Thesystem of claim 1, further comprising an accessory device coupled to thedevice, and wherein the camera orientation information comprisesrelative orientation information that includes a relative orientationbetween the camera and the accessory device.
 3. The system of claim 1,wherein the camera orientation information comprises relativeorientation information that includes a relative orientation between thecamera and a surface identified in a scene surrounding the camera. 4.The system of claim 1, wherein the camera orientation informationcomprises absolute orientation information that includes a relativeorientation between the camera and gravity.
 5. The system of claim 1,wherein the position control logic is configured to perform opticalimage stabilization while the camera is moving, during which theposition control logic temporarily moves the adjustable field of viewaway from the default field of view in response to camera motion.
 6. Thesystem of claim 1, wherein: the position control logic is configured toobtain position information associated with a target object; and theposition control logic is configured to select the lateral position ofthe default field of view using the camera orientation information andthe position information.
 7. The system of claim 6, wherein: theposition control logic is configured to identify a set of potentiallateral positions using the camera orientation information and toselect, using the position information, one of the set of potentiallateral positions as the lateral position of the default field of view.8. The system of claim 1, wherein the position control logic isconfigured to set a size of the default field of view.
 9. The system ofclaim 1, wherein: the position control logic is configured to control arelative rotation of the image sensor around the optical axis by anamount determined using the camera orientation information.
 10. A camerahaving an optical axis and an adjustable field of view, the cameracomprising: a lens; an image sensor; and a position control logicconfigured to: obtain camera position information associated with atarget object; select a lateral position of a default field of viewusing the camera position information; control a relative position ofthe lens and the image sensor to maintain the adjustable field of viewas the default field of view while the camera is stationary; and capturean image at the default field of view.
 11. The camera of claim 10,wherein the position control logic is configured to a relative tiltbetween the lens and the image sensor using the camera positioninformation.
 12. The camera of claim 10, wherein selecting the lateralposition of the default field of view comprises: determining whether aset candidate positions exists that would position the target object ina first region of the adjustable field of view.
 13. The camera of claim12, wherein selecting the lateral position of the default field of viewcomprises: selecting, in response to determining that the set ofcandidate positions exists, one of the set of candidate positions as thelateral position of the default field of view.
 14. The camera of claim12, wherein selecting the lateral position of the default field of viewcomprises: selecting, in response to determining that the set ofcandidate positions does not exist, the lateral position of the defaultfield of view at a position that places the target object in a secondregion of the adjustable field of view.
 15. A method comprising: at asystem that includes a display and a camera having an adjustable fieldof view: capturing a first image stream while the adjustable field ofview has a default field of view set at a first lateral position; andgenerating a first set of output images from the first image stream,wherein: the first lateral position is selected using camera orientationinformation associated with the camera.
 16. The method of claim 15,wherein the first set of output images is a video feed, and furthercomprising: displaying, via the display, a communication user interfacefor a videoconferencing session, the communication user interfaceincluding the video feed.
 17. The method of claim 16, wherein the videofeed includes a representation of a surface in a scene surrounding thecamera.
 18. The method of claim 15, further comprising: capturing asecond image stream while the adjustable field of view has the defaultfield of view set at a second lateral position; and generating a secondset of output images from the second image stream, wherein: the secondlateral position is selected using position information associated witha target object.
 19. The method of claim 15, wherein: the first lateralposition is selected using both the camera orientation information andposition information associated with a target object.
 20. The method ofclaim 19, further comprising: generating a second set of output imagesfrom the first image stream, wherein: the first set of output images isgenerated from a first cropping boundary applied to the first imagestream; and the second set of output images is generated from a secondcropping boundary applied to the first image stream.